Valid sequential inference on probability forecast performance

Summary Probability forecasts for binary events play a central role in many applications. Their quality is commonly assessed with proper scoring rules, which assign numerical scores such that correct forecast achieves minimal expected score. In this paper, we construct e-values testing the statistical significance of score differences competing sequential settings. E-values have been proposed as an alternative to $p$-values hypothesis testing, and they can easily be transformed into conservative by taking multiplicative inverse. The article are valid finite samples without any assumptions on data-generating processes. They also allow optional stopping, so user may decide interrupt evaluation, account available data at time, still draw statistically inference, generally not true classical $p$-value-based tests. case study post-processing precipitation forecasts, state-of-the-art dominance tests lead same conclusions.